Objectives:

Use Machine Learning To Possibly Become A Millionaire: Predicting The Stock Market? The stock market is one of the most well-known infrastructures through which anyone can potentially make a fortune. If anyone could crack the code to predicting what future stock prices are, they’ll practically rule the world.here’s just one problem. It’s pretty much impossible to accurately predict the future of the stock market.

In this project, we will work with historical data about the stock prices of a publicly listed company. We will implement a mix of machine learning algorithms to predict the future stock price of the company, starting with simple algorithm like linear regression, and then move on to advanced techniques like Auto ARIMA and Neural Networks.

Data Import

Description of fundamentals.csv

Names Description
Stocks Variable Columns Categorical - Ticker.Symbol,Period.Ending
Stocks Technical Indicator Columns

Numerical - Accounts.Payable,

Accounts.Receivable, Add.l.income.expense.items,

After.Tax.ROE, Capital.Expenditures,

Capital.Surplus, Cash.Ratio,

Cash.and.Cash.Equivalents,

Changes.in.Inventories,Common.Stocks,

Cost.of.Revenue,Current.Ratio,

Deferred.Asset.Charges,Total.Assets,

Deferred.Liability.Charges,

Depreciation,Earnings.Before.Interest.and.Tax,

Earnings.Before.Tax,Net.Income.Adjustments,

Effect.of.Exchange.Rate,

Equity.Earnings.Loss.Unconsolidated.Subsidiary,

Fixed.Assets,Goodwill,Gross.Margin,

Gross.Profit,Income.Tax,

Intangible.Assets,Interest.Expense,

Inventory,Investments,Liabilities,

Long.Term.Debt,Long.Term.Investments,

Minority.Interest,Misc..Stocks,

Net.Borrowings,Net.Cash.Flow ,

Net.Cash.Flow.Operating,Net.Cash.Flows.Financing,

Net.Cash.Flows.Investing,Net.Income,

Net.Income.Applicable.to.Common.Shareholders,

Net.Income.Cont..,Operations,

Net.Receivables,Non.Recurring.Items,

Operating.Income,Operating.Margin,

Other.Assets,Other.Current.Assets,

Other.Current.Liabilities,

Other.Equity,Other.Financing.Activities,

Other.Investing.Activities,Other.Liabilities,

Other.Operating.Activities,Pre.Tax.Margin

Other.Operating.Items,Pre.Tax.ROE,

Profit.Margin,Quick.Ratio,

Research.and.Development,

Retained.Earnings,Short.Term.Debt

Sale.and.Purchase.of.Stock,

Sales..General.and.Admin.,

Current.Portion.of.Long.Term.Debt,

Short.Term.Investments,Equity,

Total.Current.Assets,For.Year

Total.Current.Liabilities,

Total.Equity,Total.Liabilities,

Total.Revenue,Treasury.Stock,

Earnings.Per.Share,

Estimated.Shares.Outstanding

Description of securities.csv

Names Description
Ticker.symbol Categorical - Ticker.Symbol
GICS.Sector Categorical -Industry of the ticker
GICS.Sub.Industry Categorical -Sub.Industry of the ticker

Description of sprice.csv

Names Description
Date Date - Date for which prices are recorded
Symbol Categorical - Ticker.Symbol
Open Numerical - Opening price of the stock on a particular day
Close Numerical - Closing price of the stock on a particular day
Low Numerical -Low price of the stock on a particular day
High Numerical -High price of the stock on a particular day
Volume Numerical -Volume of the stock on a particular day

Data Structure and manipulation.

Names Description
Ticker.Symbol Categorical - Stock Symbol
Industry Categorical - Idustry of the Stock
endyear Date - Year of the revenue
quarter Date - Quarter of the revenue
Stock.p Numerical - Mean of Close price
Capital.Expenditures Numerical - Technical Indicator
Cash.Ratio Numerical - Technical Indicator
Cost.of.Revenue Numerical - Technical Indicator
Depreciation Numerical - Technical Indicator
Earnings.Before.Interest.and.Tax Numerical - Technical Indicator
Effect.of.Exchange.Rate Numerical - Technical Indicator
Fixed.Assets Numerical - Technical Indicator
Gross.Profit Numerical - Technical Indicator
Investments Numerical - Technical Indicator
Liabilities Numerical - Technical Indicator
Long.Term.Debt Numerical - Technical Indicator
Long.Term.Investments Numerical - Technical Indicator
Net.Cash.Flow Numerical - Technical Indicator
Net.Income Numerical - Technical Indicator
Pre.Tax.ROE Numerical - Technical Indicator
Profit.Margin Numerical - Technical Indicator
Retained.Earnings Numerical - Technical Indicator
Total.Current.Assets Numerical - Technical Indicator
Total.Current.Liabilities Numerical - Technical Indicator
Total.Equity Numerical - Technical Indicator
Total.Liabilities Numerical - Technical Indicator
Total.Revenue Numerical - Technical Indicator
Earnings.Per.Share Numerical - Technical Indicator
Estimated.Shares.Outstanding Numerical - Technical Indicator

Data Science Analysis

Corrplot With Correlation Coefficient

Numerical Summaries

Frequency Table for Industry:

Consumer Discretionary       Consumer Staples                 Energy 
                   120                     50                     58 
            Financials            Health Care            Industrials 
                    10                     74                    120 
Information Technology              Materials            Real Estate 
                   108                     44                     14 
             Utilities 
                    42 
Frequency Table for MarketCapIndiactor:

  high    low medium 
   292    301     47 
Contingency Table for Industry/ MarketCapIndiactor:
                        
                         high low medium
  Consumer Discretionary   46  67      7
  Consumer Staples         28  18      4
  Energy                   32  23      3
  Financials                3   4      3
  Health Care              45  23      6
  Industrials              47  65      8
  Information Technology   49  52      7
  Materials                17  24      3
  Real Estate               6   6      2
  Utilities                19  19      4
Proportions Table for Industry/ MarketCapIndiactor:
                        
                              high       low    medium
  Consumer Discretionary 0.0718750 0.1046875 0.0109375
  Consumer Staples       0.0437500 0.0281250 0.0062500
  Energy                 0.0500000 0.0359375 0.0046875
  Financials             0.0046875 0.0062500 0.0046875
  Health Care            0.0703125 0.0359375 0.0093750
  Industrials            0.0734375 0.1015625 0.0125000
  Information Technology 0.0765625 0.0812500 0.0109375
  Materials              0.0265625 0.0375000 0.0046875
  Real Estate            0.0093750 0.0093750 0.0031250
  Utilities              0.0296875 0.0296875 0.0062500
Summary MarketCapIndiactor
   Length     Class      Mode 
      640 character character 
Summary Industry
Consumer Discretionary       Consumer Staples                 Energy 
                   120                     50                     58 
            Financials            Health Care            Industrials 
                    10                     74                    120 
Information Technology              Materials            Real Estate 
                   108                     44                     14 
             Utilities 
                    42 
Numerical Summary of Market Cap
     Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
2.609e+09 1.030e+10 1.678e+10 3.534e+10 3.393e+10 6.907e+11 
Numerical Summary of Market Cap grouped by Industry
# A tibble: 10 x 6
   Industry                 sum         mean        min          max      median
   <fct>                  <dbl>        <dbl>      <dbl>        <dbl>       <dbl>
 1 Consumer Discretion… 3.33e12 27755715066.     4.62e9      2.23e11     1.38e10
 2 Consumer Staples     2.74e12 54777333131.     4.69e9      2.53e11     2.83e10
 3 Energy               2.44e12 42025301685.     4.57e9      4.16e11     2.27e10
 4 Financials           1.55e11 15499103508.     7.49e9      2.69e10     1.60e10
 5 Health Care          3.31e12 44710159546.     3.22e9      2.08e11     2.31e10
 6 Industrials          3.04e12 25355115660.     4.07e9      1.01e11     1.38e10
 7 Information Technol… 5.52e12 51151898108.     2.61e9      6.91e11     1.61e10
 8 Materials            9.05e11 20569895953.     4.04e9      5.99e10     1.44e10
 9 Real Estate          3.33e11 23773975192.     5.95e9      4.93e10     1.81e10
10 Utilities            8.41e11 20019656246.     3.28e9      5.23e10     1.63e10
[1] "skewness  of Industry -0.051643"
[1] "skewness  of MarketCapIndiactor 0.476626"
[1] "skewness  of Market.Cap 5.468953"
[1] "skewness  of Total.Revenue 6.125020"
[1] "skewness  of Gross.Profit 5.369591"
Shapiro Wilkes Test

    Shapiro-Wilk normality test

data:  as.numeric(as.factor(s$Industry))[0:200]
W = 0.90934, p-value = 1.033e-09

###Graphical Summaries

Correlation and Regression

'data.frame':   640 obs. of  29 variables:
 $ Ticker.Symbol                   : Factor w/ 320 levels "AAL","AAPL","ABBV",..: 1 1 2 2 3 3 4 4 5 5 ...
 $ Capital.Expenditures            : num  -5.31e+09 -6.15e+09 -9.57e+09 -1.12e+10 -6.12e+08 ...
 $ Cash.Ratio                      : num  60 51 40 52 74 77 10 10 43 67 ...
 $ Cost.of.Revenue                 : num  1.56e+10 1.11e+10 1.12e+11 1.40e+11 4.43e+09 ...
 $ Depreciation                    : num  1.34e+09 1.49e+09 7.95e+09 1.13e+10 7.86e+08 ...
 $ Earnings.Before.Interest.and.Tax: num  4.10e+09 5.50e+09 5.35e+10 7.25e+10 2.76e+09 ...
 $ Fixed.Assets                    : num  2.31e+10 2.75e+10 2.06e+10 2.25e+10 2.48e+09 ...
 $ Gross.Profit                    : num  2.70e+10 2.99e+10 7.05e+10 9.36e+10 1.55e+10 ...
 $ Investments                     : num  1.80e+09 4.43e+08 -9.03e+09 -4.44e+10 3.08e+08 ...
 $ Liabilities                     : num  -1.03e+09 -6.33e+08 1.34e+10 1.52e+10 -1.93e+08 ...
 $ Long.Term.Debt                  : num  1.60e+10 1.83e+10 2.90e+10 5.33e+10 1.05e+10 ...
 $ Net.Cash.Flow                   : num  -1.46e+08 -6.04e+08 -4.15e+08 7.28e+09 -1.25e+09 ...
 $ Pre.Tax.ROE                     : num  159 82 48 61 136 168 34 44 12 15 ...
 $ Profit.Margin                   : num  7 19 22 23 9 23 0 0 11 22 ...
 $ Retained.Earnings               : num  -8.56e+09 -1.23e+09 8.72e+10 9.23e+10 5.35e+08 ...
 $ Total.Current.Assets            : num  1.18e+10 9.98e+09 6.85e+10 8.94e+10 1.61e+10 ...
 $ Total.Current.Liabilities       : num  1.34e+10 1.36e+10 6.34e+10 8.06e+10 1.14e+10 ...
 $ Total.Equity                    : num  2.02e+09 5.64e+09 1.12e+11 1.19e+11 1.74e+09 ...
 $ Total.Liabilities               : num  4.12e+10 4.28e+10 1.20e+11 1.71e+11 2.58e+10 ...
 $ Total.Revenue                   : num  4.26e+10 4.10e+10 1.83e+11 2.34e+11 2.00e+10 ...
 $ Earnings.Per.Share              : num  4.02 11.39 6.49 9.28 1.11 ...
 $ Estimated.Shares.Outstanding    : num  7.17e+08 6.68e+08 6.09e+09 5.75e+09 1.60e+09 ...
 $ Industry                        : Factor w/ 10 levels "Consumer Discretionary",..: 6 6 7 7 5 5 5 5 5 5 ...
 $ endyear                         : Factor w/ 2 levels "2014","2015": 1 2 1 2 1 2 1 2 1 2 ...
 $ quarter                         : Factor w/ 4 levels "1","2","3","4": 4 4 3 3 4 4 3 3 4 4 ...
 $ Stock.p                         : num  38.9 45.1 92.3 120 55.5 ...
 $ Year                            : Factor w/ 2 levels "2014","2015": 1 2 1 2 1 2 1 2 1 2 ...
 $ Market.Cap                      : num  2.79e+10 3.01e+10 5.62e+11 6.91e+11 8.86e+10 ...
 $ MarketCapIndiactor              : chr  "high" "high" "high" "high" ...


    Welch Two Sample t-test

data:  visualization_data$Market.Cap and visualization_data$Total.Revenue
t = 4.908, df = 1179.8, p-value = 1.049e-06
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
  8411277703 19614850898
sample estimates:
  mean of x   mean of y 
35337997368 21324933067 
[1] "P value for Market cap & Total revenue 0.000001"
[1] "Group Means for Market cap & Total revenue 35337997367.965248"
[2] "Group Means for Market cap & Total revenue 21324933067.187500"
[1] "Confidence interval for difference for Market cap & Total revenue 8411277703.442698" 
[2] "Confidence interval for difference for Market cap & Total revenue 19614850898.112801"
[1] "Confidence level for Market cap & Total revenue 0.950000"

Inferential Statistics

[1] "GOOGL"

Hypothesis Test, Mean Change Price


    One Sample t-test

data:  DataSetGoogle$Price2012
t = -33.718, df = 501, p-value < 2.2e-16
alternative hypothesis: true mean is not equal to 6.241242
95 percent confidence interval:
 -0.1912902  0.5170744
sample estimates:
mean of x 
0.1628921 

Paired Tests


    F test to compare two variances

data:  DataSetGoogle$Price2012 and DataSetGoogle$Price2014
F = 0.32317, num df = 501, denom df = 501, p-value < 2.2e-16
alternative hypothesis: true ratio of variances is not equal to 1
95 percent confidence interval:
 0.2711993 0.3851034
sample estimates:
ratio of variances 
         0.3231715 

    Two Sample t-test

data:  DataSetGoogle$Price2012 and DataSetGoogle$Price2014
t = 1.6032, df = 1002, p-value = 0.1092
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.1310177  1.3005862
sample estimates:
 mean of x  mean of y 
 0.1628921 -0.4218921 

    Welch Two Sample t-test

data:  DataSetGoogle$Price2012 and DataSetGoogle$Price2014
t = 1.6032, df = 794.2, p-value = 0.1093
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.1312443  1.3008127
sample estimates:
 mean of x  mean of y 
 0.1628921 -0.4218921 

Independent Tests

[1] "GOOGL"
[1] "AMZN"

    Two Sample t-test

data:  pairedtestdataset$PriceGoogle and pairedtestdataset$PriceAmazon
t = -0.41406, df = 1002, p-value = 0.6789
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.3699250  0.2410143
sample estimates:
  mean of x   mean of y 
-0.12885399 -0.06439866 

    Welch Two Sample t-test

data:  pairedtestdataset$PriceGoogle and pairedtestdataset$PriceAmazon
t = -0.41406, df = 828.45, p-value = 0.6789
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.3700024  0.2410917
sample estimates:
  mean of x   mean of y 
-0.12885399 -0.06439866 

    F test to compare two variances

data:  pairedtestdataset$PriceGoogle and pairedtestdataset$PriceAmazon
F = 2.688, num df = 501, denom df = 501, p-value < 2.2e-16
alternative hypothesis: true ratio of variances is not equal to 1
95 percent confidence interval:
 2.255708 3.203109
sample estimates:
ratio of variances 
          2.687988 

Regression Diagnostic Plots

MultiCollinearity

[1] "VIF score for all features"
                    Total.Equity                    Total.Revenue 
                        3.198600                         3.939829 
    Estimated.Shares.Outstanding Earnings.Before.Interest.and.Tax 
                        2.732357                         3.040498 
                    Gross.Profit                          Stock.p 
                        7.184466                         1.062775 

Regression- Find if Market.Cap is affected by trading technical indicators

Linear Regression

  • Objective: Check if Market.Cap is affected by “Total.Equity”,“Total.Revenue”, “Estimated.Shares.Outstanding”,“Earnings.Before.Interest.and.Tax”,“Gross.Profit”,“Stock.p”.
  • Import the dataset, and clean the dataset for target and feature variables.
  • Data slicing:
    • Dataset is split into 75 percent of training data, 25 % of test set.
  • Preprocessing:
    • We scale and center the predictor & feature variables before passing to models.
  • Training the Multiple Linear Regression:
    • Training the Multiple Linear Regression with metric as “RMSE”.
    • train() method is passed with repeated cross-validation resampling method for 10 number of resampling iterations repeated for 3 times.The default method for optimizing tuning parameters in train is to use a grid search.Hyperparameter Tuning using Grid Search Cross Validation
  • Trained classifier results
    • We can check the result of our train() method by a print fit.LR variable.
+ Fold1.Rep1: intercept=TRUE 
- Fold1.Rep1: intercept=TRUE 
+ Fold2.Rep1: intercept=TRUE 
- Fold2.Rep1: intercept=TRUE 
+ Fold3.Rep1: intercept=TRUE 
- Fold3.Rep1: intercept=TRUE 
+ Fold1.Rep2: intercept=TRUE 
- Fold1.Rep2: intercept=TRUE 
+ Fold2.Rep2: intercept=TRUE 
- Fold2.Rep2: intercept=TRUE 
+ Fold3.Rep2: intercept=TRUE 
- Fold3.Rep2: intercept=TRUE 
+ Fold1.Rep3: intercept=TRUE 
- Fold1.Rep3: intercept=TRUE 
+ Fold2.Rep3: intercept=TRUE 
- Fold2.Rep3: intercept=TRUE 
+ Fold3.Rep3: intercept=TRUE 
- Fold3.Rep3: intercept=TRUE 
Aggregating results
Fitting final model on full training set

Call:
lm(formula = .outcome ~ ., data = dat)

Residuals:
       Min         1Q     Median         3Q        Max 
-2.233e+10 -4.448e+09 -9.596e+08  2.379e+09  3.628e+10 

Coefficients:
                                   Estimate Std. Error t value Pr(>|t|)    
(Intercept)                       2.139e+10  2.102e+08 101.771  < 2e-16 ***
Total.Equity                      9.649e+08  2.724e+08   3.542 0.000409 ***
Total.Revenue                    -5.398e+07  3.301e+08  -0.164 0.870121    
Estimated.Shares.Outstanding      4.801e+09  3.153e+08  15.226  < 2e-16 ***
Earnings.Before.Interest.and.Tax  3.977e+09  3.150e+08  12.623  < 2e-16 ***
Gross.Profit                      3.104e+09  3.696e+08   8.397  < 2e-16 ***
Stock.p                           3.654e+09  2.338e+08  15.632  < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 8.237e+09 on 1529 degrees of freedom
Multiple R-squared:  0.6548,    Adjusted R-squared:  0.6534 
F-statistic: 483.3 on 6 and 1529 DF,  p-value: < 2.2e-16
  • Prediction
    • We are ready to predict classes for our test set using predict() method, and calculate error rate and rmse.We get Multiple R-squared value as 0.99499149, and error rate as 0.2309932.
RMSE:  4968177423 Error:  0.2309932 R2:  0.9949914
  • Interpretation of results

    • R-squared has the useful property that its scale is intuitive: it ranges from zero to one, with zero indicating that the proposed model does not improve prediction over the mean model, and one indicating perfect prediction. Improvement in the regression model results in proportional increases in R-squared.

    • The RMSE is the square root of the variance of the residuals. It indicates the absolute fit of the model to the data–how close the observed data points are to the model’s predicted values. Whereas R-squared is a relative measure of fit, RMSE is an absolute measure of fit. Lower values of RMSE indicate better fit.

    • Since error percentage is 23, R2=0.99, RMSE is , it is safe to assume that market cap is affected by “Total.Equity”,“Total.Revenue”, “Estimated.Shares.Outstanding”,“Earnings.Before.Interest.and.Tax”,“Gross.Profit”,“Stock.p”.

Polynominal Regression

  • Polynomial regression is a special case of linear regression where we fit a polynomial equation on the data with a curvilinear relationship between the target variable and the independent variables.

  • In a curvilinear relationship, the value of the target variable changes in a non-uniform manner with respect to the predictor (s).

  • poly function returns or evaluates orthogonal polynomials of degree 1 to degree over the specified set of points x: these are all orthogonal to the constant polynomial of degree 0.

+ Fold1.Rep1: intercept=TRUE 
- Fold1.Rep1: intercept=TRUE 
+ Fold2.Rep1: intercept=TRUE 
- Fold2.Rep1: intercept=TRUE 
+ Fold3.Rep1: intercept=TRUE 
- Fold3.Rep1: intercept=TRUE 
+ Fold1.Rep2: intercept=TRUE 
- Fold1.Rep2: intercept=TRUE 
+ Fold2.Rep2: intercept=TRUE 
- Fold2.Rep2: intercept=TRUE 
+ Fold3.Rep2: intercept=TRUE 
- Fold3.Rep2: intercept=TRUE 
+ Fold1.Rep3: intercept=TRUE 
- Fold1.Rep3: intercept=TRUE 
+ Fold2.Rep3: intercept=TRUE 
- Fold2.Rep3: intercept=TRUE 
+ Fold3.Rep3: intercept=TRUE 
- Fold3.Rep3: intercept=TRUE 
Aggregating results
Fitting final model on full training set

Call:
lm(formula = .outcome ~ ., data = dat)

Residuals:
       Min         1Q     Median         3Q        Max 
-1.693e+10 -3.719e+09 -1.128e+09  1.712e+09  4.125e+10 

Coefficients:
                                               Estimate Std. Error t value
(Intercept)                                   2.139e+10  1.893e+08 112.981
`poly(Total.Equity, 2)1`                      6.920e+08  2.562e+08   2.701
`poly(Total.Equity, 2)2`                     -3.021e+08  2.006e+08  -1.506
`poly(Total.Revenue, 2)1`                    -1.022e+09  3.300e+08  -3.096
`poly(Total.Revenue, 2)2`                     4.368e+08  2.210e+08   1.976
`poly(Estimated.Shares.Outstanding, 2)1`      6.839e+09  3.241e+08  21.099
`poly(Estimated.Shares.Outstanding, 2)2`     -3.001e+09  2.174e+08 -13.804
`poly(Earnings.Before.Interest.and.Tax, 2)1`  2.823e+09  3.021e+08   9.345
`poly(Earnings.Before.Interest.and.Tax, 2)2`  7.322e+08  2.015e+08   3.633
`poly(Gross.Profit, 2)1`                      2.982e+09  3.786e+08   7.876
`poly(Gross.Profit, 2)2`                     -7.289e+08  2.282e+08  -3.195
`poly(Stock.p, 2)1`                           4.900e+09  2.256e+08  21.724
`poly(Stock.p, 2)2`                          -2.355e+09  2.020e+08 -11.659
                                             Pr(>|t|)    
(Intercept)                                   < 2e-16 ***
`poly(Total.Equity, 2)1`                      0.00700 ** 
`poly(Total.Equity, 2)2`                      0.13233    
`poly(Total.Revenue, 2)1`                     0.00199 ** 
`poly(Total.Revenue, 2)2`                     0.04833 *  
`poly(Estimated.Shares.Outstanding, 2)1`      < 2e-16 ***
`poly(Estimated.Shares.Outstanding, 2)2`      < 2e-16 ***
`poly(Earnings.Before.Interest.and.Tax, 2)1`  < 2e-16 ***
`poly(Earnings.Before.Interest.and.Tax, 2)2`  0.00029 ***
`poly(Gross.Profit, 2)1`                     6.37e-15 ***
`poly(Gross.Profit, 2)2`                      0.00143 ** 
`poly(Stock.p, 2)1`                           < 2e-16 ***
`poly(Stock.p, 2)2`                           < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 7.42e+09 on 1523 degrees of freedom
Multiple R-squared:  0.721, Adjusted R-squared:  0.7188 
F-statistic: 327.9 on 12 and 1523 DF,  p-value: < 2.2e-16
  • Polynomial Regression Prediction & Accuracy
RMSE:  5761789130 Error:  0.2309932 R2:  0.9938427

Spline Regression

  • Spline is a special function defined piece-wise by polynomials. The term “spline” is used to refer to a wide class of functions that are used in applications requiring data interpolation and/or smoothing. The data may be either one-dimensional or multi-dimensional.
  • Spline Regression is one of the non-parametric regression technique. In this technique the dataset is divided into bins at intervals or points which we called as knots. Also this bin has its separate fit.
  • The disadvantages of the polynomial regression can be overcome by using Spline Regression. Polynomial regression only captures a certain amount of curvature in a nonlinear relationship. An alternative, and often superior, approach to modeling nonlinear relationships is to use splines.
  • Splines provide a way to smoothly interpolate between fixed points, called knots. Polynomial regression is computed between knots. In other words, splines are series of polynomial segments strung together, joining at knots.
  • The generic function quantile produces sample quantiles corresponding to the given probabilities.
  • bs {splines} Generate the B-spline basis matrix for a polynomial spline.bs uses knots which is the internal breakpoints that define the spline.Typical values are the mean or median for one knot, quantiles for more knots.
'data.frame':   640 obs. of  23 variables:
 $ Capital.Expenditures            : num  -5.31e+09 -6.15e+09 -9.57e+09 -1.12e+10 -6.12e+08 ...
 $ Cash.Ratio                      : num  60 51 40 52 74 77 10 10 43 67 ...
 $ Cost.of.Revenue                 : num  1.56e+10 1.11e+10 1.40e+10 1.40e+10 4.43e+09 ...
 $ Depreciation                    : num  1.34e+09 1.49e+09 1.14e+09 1.14e+09 7.86e+08 ...
 $ Earnings.Before.Interest.and.Tax: num  4.10e+09 2.43e+09 2.43e+09 2.43e+09 2.76e+09 ...
 $ Fixed.Assets                    : num  2.31e+10 9.51e+09 2.06e+10 2.25e+10 2.48e+09 ...
 $ Gross.Profit                    : num  7.33e+09 7.33e+09 7.33e+09 7.33e+09 7.33e+09 ...
 $ Investments                     : num  1.80e+09 4.43e+08 -9.03e+09 -4.44e+10 3.08e+08 ...
 $ Liabilities                     : num  1.41e+08 1.41e+08 1.41e+08 1.41e+08 -1.93e+08 ...
 $ Long.Term.Debt                  : num  1.60e+10 6.61e+09 6.61e+09 6.61e+09 1.05e+10 ...
 $ Net.Cash.Flow                   : num  -1.46e+08 -6.04e+08 -4.15e+08 7.28e+09 -1.25e+09 ...
 $ Pre.Tax.ROE                     : num  159 82 48 61 136 168 34 44 12 15 ...
 $ Profit.Margin                   : num  7 19 22 23 9 23 0 0 11 22 ...
 $ Retained.Earnings               : num  -8.56e+09 -1.23e+09 9.73e+09 9.73e+09 5.35e+08 ...
 $ Total.Current.Assets            : num  1.18e+10 9.98e+09 6.85e+10 8.94e+10 1.61e+10 ...
 $ Total.Current.Liabilities       : num  1.34e+10 1.36e+10 6.34e+10 8.06e+10 1.14e+10 ...
 $ Total.Equity                    : num  2.02e+09 5.64e+09 9.41e+09 9.41e+09 1.74e+09 ...
 $ Total.Liabilities               : num  4.12e+10 4.28e+10 1.69e+10 1.69e+10 2.58e+10 ...
 $ Total.Revenue                   : num  2.13e+10 2.13e+10 2.13e+10 2.13e+10 2.00e+10 ...
 $ Earnings.Per.Share              : num  4.02 3.27 6.49 3.27 1.11 ...
 $ Estimated.Shares.Outstanding    : num  7.17e+08 6.68e+08 5.64e+08 5.64e+08 5.64e+08 ...
 $ Stock.p                         : num  38.9 45.1 92.3 120 55.5 ...
 $ Market.Cap                      : num  2.79e+10 3.01e+10 3.53e+10 3.53e+10 3.53e+10 ...

Call:
lm(formula = Market.Cap ~ bs(Total.Equity, knots = knots) + bs(Total.Revenue, 
    knots = knots) + bs(Estimated.Shares.Outstanding, knots = knots) + 
    bs(Earnings.Before.Interest.and.Tax, knots = knots) + bs(Gross.Profit, 
    knots = knots), data = splineDatasetSubTrain)

Residuals:
       Min         1Q     Median         3Q        Max 
-2.800e+10 -4.215e+09 -6.977e+08  2.852e+09  3.430e+10 

Coefficients: (12 not defined because of singularities)
                                                       Estimate Std. Error
(Intercept)                                           3.427e+10  6.318e+09
bs(Total.Equity, knots = knots)1                     -1.727e+10  6.381e+09
bs(Total.Equity, knots = knots)2                     -5.320e+09  3.911e+09
bs(Total.Equity, knots = knots)3                     -8.529e+09  5.238e+09
bs(Total.Equity, knots = knots)4                     -8.703e+09  4.685e+09
bs(Total.Equity, knots = knots)5                     -9.102e+09  5.085e+09
bs(Total.Equity, knots = knots)6                             NA         NA
bs(Total.Equity, knots = knots)7                             NA         NA
bs(Total.Revenue, knots = knots)1                    -2.704e+09  2.740e+09
bs(Total.Revenue, knots = knots)2                    -7.207e+09  1.720e+09
bs(Total.Revenue, knots = knots)3                    -5.667e+09  2.841e+09
bs(Total.Revenue, knots = knots)4                    -2.484e+09  3.860e+09
bs(Total.Revenue, knots = knots)5                    -5.955e+09  5.374e+09
bs(Total.Revenue, knots = knots)6                    -1.659e+10  4.956e+09
bs(Total.Revenue, knots = knots)7                            NA         NA
bs(Estimated.Shares.Outstanding, knots = knots)1      1.295e+09  2.767e+09
bs(Estimated.Shares.Outstanding, knots = knots)2      1.918e+10  2.905e+09
bs(Estimated.Shares.Outstanding, knots = knots)3      9.242e+08  2.432e+09
bs(Estimated.Shares.Outstanding, knots = knots)4     -5.107e+09  5.035e+09
bs(Estimated.Shares.Outstanding, knots = knots)5             NA         NA
bs(Estimated.Shares.Outstanding, knots = knots)6             NA         NA
bs(Estimated.Shares.Outstanding, knots = knots)7             NA         NA
bs(Earnings.Before.Interest.and.Tax, knots = knots)1 -4.189e+10  8.942e+09
bs(Earnings.Before.Interest.and.Tax, knots = knots)2  1.181e+10  3.244e+09
bs(Earnings.Before.Interest.and.Tax, knots = knots)3 -3.521e+08  5.359e+09
bs(Earnings.Before.Interest.and.Tax, knots = knots)4  3.307e+09  1.027e+10
bs(Earnings.Before.Interest.and.Tax, knots = knots)5         NA         NA
bs(Earnings.Before.Interest.and.Tax, knots = knots)6         NA         NA
bs(Earnings.Before.Interest.and.Tax, knots = knots)7         NA         NA
bs(Gross.Profit, knots = knots)1                      1.185e+10  2.733e+09
bs(Gross.Profit, knots = knots)2                      1.652e+10  2.554e+09
bs(Gross.Profit, knots = knots)3                      1.585e+10  2.959e+09
bs(Gross.Profit, knots = knots)4                      1.685e+10  2.868e+09
bs(Gross.Profit, knots = knots)5                             NA         NA
bs(Gross.Profit, knots = knots)6                             NA         NA
bs(Gross.Profit, knots = knots)7                             NA         NA
                                                     t value Pr(>|t|)    
(Intercept)                                            5.424 6.77e-08 ***
bs(Total.Equity, knots = knots)1                      -2.706 0.006888 ** 
bs(Total.Equity, knots = knots)2                      -1.360 0.173932    
bs(Total.Equity, knots = knots)3                      -1.628 0.103653    
bs(Total.Equity, knots = knots)4                      -1.858 0.063417 .  
bs(Total.Equity, knots = knots)5                      -1.790 0.073660 .  
bs(Total.Equity, knots = knots)6                          NA       NA    
bs(Total.Equity, knots = knots)7                          NA       NA    
bs(Total.Revenue, knots = knots)1                     -0.987 0.323903    
bs(Total.Revenue, knots = knots)2                     -4.190 2.95e-05 ***
bs(Total.Revenue, knots = knots)3                     -1.995 0.046232 *  
bs(Total.Revenue, knots = knots)4                     -0.643 0.520074    
bs(Total.Revenue, knots = knots)5                     -1.108 0.267934    
bs(Total.Revenue, knots = knots)6                     -3.346 0.000839 ***
bs(Total.Revenue, knots = knots)7                         NA       NA    
bs(Estimated.Shares.Outstanding, knots = knots)1       0.468 0.639956    
bs(Estimated.Shares.Outstanding, knots = knots)2       6.602 5.59e-11 ***
bs(Estimated.Shares.Outstanding, knots = knots)3       0.380 0.703998    
bs(Estimated.Shares.Outstanding, knots = knots)4      -1.014 0.310574    
bs(Estimated.Shares.Outstanding, knots = knots)5          NA       NA    
bs(Estimated.Shares.Outstanding, knots = knots)6          NA       NA    
bs(Estimated.Shares.Outstanding, knots = knots)7          NA       NA    
bs(Earnings.Before.Interest.and.Tax, knots = knots)1  -4.684 3.06e-06 ***
bs(Earnings.Before.Interest.and.Tax, knots = knots)2   3.640 0.000282 ***
bs(Earnings.Before.Interest.and.Tax, knots = knots)3  -0.066 0.947627    
bs(Earnings.Before.Interest.and.Tax, knots = knots)4   0.322 0.747503    
bs(Earnings.Before.Interest.and.Tax, knots = knots)5      NA       NA    
bs(Earnings.Before.Interest.and.Tax, knots = knots)6      NA       NA    
bs(Earnings.Before.Interest.and.Tax, knots = knots)7      NA       NA    
bs(Gross.Profit, knots = knots)1                       4.336 1.54e-05 ***
bs(Gross.Profit, knots = knots)2                       6.471 1.31e-10 ***
bs(Gross.Profit, knots = knots)3                       5.355 9.87e-08 ***
bs(Gross.Profit, knots = knots)4                       5.875 5.18e-09 ***
bs(Gross.Profit, knots = knots)5                          NA       NA    
bs(Gross.Profit, knots = knots)6                          NA       NA    
bs(Gross.Profit, knots = knots)7                          NA       NA    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 8.429e+09 on 1512 degrees of freedom
Multiple R-squared:  0.6425,    Adjusted R-squared:  0.637 
F-statistic: 118.1 on 23 and 1512 DF,  p-value: < 2.2e-16
  • Spline Regression Prediction & Accuracy
RMSE:  4626494209 Error:  0.2151068 R2:  0.8751914

Generalized Linear Model

Why Use GAM?Relationships between the individual predictors and the dependent variable follow smooth patterns that can be linear or nonlinear. + Mathematically speaking, GAM is an additive modeling technique where the impact of the predictive variables is captured through smooth functions which—depending on the underlying patterns in the data—can be nonlinear. + GAM can capture common nonlinear patterns that a classic linear model would miss. + GAM framework allows us to control smoothness of the predictor functions to prevent overfitting. By controlling the wiggliness of the predictor functions, we can directly tackle the bias/variance tradeoff.


Family: gaussian 
Link function: identity 

Formula:
Market.Cap ~ Total.Equity + Earnings.Before.Interest.and.Tax

Parametric coefficients:
                                  Estimate Std. Error t value Pr(>|t|)    
(Intercept)                      2.173e-09  5.850e-11   37.14   <2e-16 ***
Total.Equity                     1.052e+00  5.873e-02   17.92   <2e-16 ***
Earnings.Before.Interest.and.Tax 8.172e+00  2.327e-01   35.11   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1


Rank: 2/3
R-sq.(adj) =  0.461   Deviance explained = 43.4%
GCV = 1.1162e+20  Scale est. = 1.1147e+20  n = 1536
  • GAM Regression Prediction & Accuracy
RMSE:  6937151743 Error:  0.354476 R2:  0.6515361
  • GAM Scatter Plot 3D Visualization

  • Visualization part of gam model

  • Diagnosing GAM model issues


Method: GCV   Optimizer: magic
Model required no smoothing parameter selectionModel rank =  2 / 3 

Penalized Cubic Regression Spline

  • Cubic Regression Spline is specified by bs=“cr”. These have a cubic spline basis defined by a modest sized set of knots spread evenly through the covariate values. They are penalized by the conventional intergrated square second derivative cubic spline penalty.

Family: gaussian 
Link function: identity 

Formula:
Market.Cap ~ s(Total.Equity, bs = "cr") + s(Total.Revenue, bs = "cr") + 
    s(Earnings.Before.Interest.and.Tax, bs = "cr")

Parametric coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept) 2.144e+10  3.458e+08      62   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Approximate significance of smooth terms:
                                      edf Ref.df      F  p-value    
s(Total.Equity)                     8.091  8.625  6.858  < 2e-16 ***
s(Total.Revenue)                    4.292  5.027  4.697 0.000331 ***
s(Earnings.Before.Interest.and.Tax) 8.418  8.829 22.214  < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

R-sq.(adj) =  0.607   Deviance explained =   62%
GCV = 7.9235e+19  Scale est. = 7.6536e+19  n = 640
  • Diagnosing Penalized Cubic Regression Spline GAM model issues


Method: GCV   Optimizer: magic
Smoothing parameter selection converged after 7 iterations.
The RMS GCV score gradient at convergence was 2.367604e+13 .
The Hessian was positive definite.
Model rank =  28 / 28 

Basis dimension (k) checking results. Low p-value (k-index<1) may
indicate that k is too low, especially if edf is close to k'.

                                      k'  edf k-index p-value  
s(Total.Equity)                     9.00 8.09    1.07   0.956  
s(Total.Revenue)                    9.00 4.29    0.95   0.088 .
s(Earnings.Before.Interest.and.Tax) 9.00 8.42    1.03   0.798  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

  • Cubic Regression Spline Prediction & Accuracy
RMSE:  5328801306 Error:  0.2722922 R2:  0.9062193

Extreme Gradient Boosting

  • XGBoost is an implementation of the Gradient Boosted Decision Trees algorithm.

  • XGBoost supports various objective functions, including regression, classification and ranking.

  • XGBoost gives among the best performances in many machine learning applications. It is optimized gradient-boosting machine learning library. The core algorithm is parallelizable and hence it can use all the processing power of your machine and the machines in your cluster. In R, according to the package documentation, since the package can automatically do parallel computation on a single machine, it could be more than 10 times faster than existing gradient boosting packages.

  • XGBoost shines when we have lots of training data where the features are numeric or mixture of numeric and categorical fields. It is also important to note that xgboost is not the best algorithm out there when all the features are categorical or when the number of rows is less than the number of fields (columns).

  • The data argument in the xgboost R function is for the input features dataset. It accepts a matrix, dgCMatrix, or local data file. The nrounds argument refers to the max number of iterations (i.e. the number of trees added to the model).

  • There are different hyperparameters that we can tune and the parametres are different from baselearner to baselearner. In tree based learners, which are the most common ones in xgboost applications, the following are the most commonly tuned hyperparameters:

  • learning rate: learning rate/eta- governs how quickly the model fits the residual error using additional base learners. If it is a smaller learning rate, it will need more boosting rounds, hence more time, to achieve the same reduction in residual error as one with larger learning rate. Typically, it lies between 0.01 - 0.3

  • The three hyperparameters below are regularization hyperparameters.

  • gamma: min loss reduction to create new tree split. default = 0 means no regularization.

  • lambda: L2 reg on leaf weights. Equivalent to Ridge regression.

  • alpha: L1 reg on leaf weights. Equivalent to Lasso regression.

  • max_depth: max depth per tree. This controls how deep our tree can grow. The Larger the depth, more complex the model will be and higher chances of overfitting. Larger data sets require deep trees to learn the rules from data. Default = 6.

  • subsample: % samples used per tree. This is the fraction of the total training set that can be used in any boosting round. Low value may lead to underfitting issues. A very high value can cause over-fitting problems.

  • colsample_bytree: % features used per tree. This is the fraction of the number of columns that we can use in any boosting round. A smaller value is an additional regularization and a larger value may be cause overfitting issues.

  • n_estimators: number of estimators (base learners). This is the number of boosting rounds.

[17:36:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:10] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:10] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:12] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:12] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:22] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:22] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:22] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:23] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:23] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:24] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:24] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:24] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:10] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:10] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:10] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:12] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:12] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:22] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:23] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.

  • Extreme Gradient Boosting Prediction & Accuracy
MSE:  3.852715e+20 MAE:  18121969088  RMSE:  19628334514

[1] 3988987596
[1] 0.2038301
[1] 0.8701342
RMSE:  3988987596 Error:  0.2038301 R2:  0.8701342

Compare Regression Models

X.Algorithm. X.RMSE. X.R2. X.Error.
Algorithm RMSE R2 Error
ExtremeGradinetBoosting 3988987596.3155 0.870134218809127 0.203830128687742
Cubic Regression Spline 5328801306.20932 0.906219318025209 0.272292211938517
GAM 6937151742.56324 0.651536114257158 0.354476041419414
Spline Regression 4626494208.87458 0.875191434305949 0.215106804884498
PolynomialRegression 0.99384270252873 0.230993214992809 5761789129.92617

Conclusion

As such, we provide evidence suggesting that stock technical indicator has influence in stock market market cap. PolynomialRegression outperforms other model with lesser error and with R2=0.994 the model completely fit.

Generalized Linear Model

Why Use GAM?Relationships between the individual predictors and the dependent variable follow smooth patterns that can be linear or nonlinear. + Mathematically speaking, GAM is an additive modeling technique where the impact of the predictive variables is captured through smooth functions which—depending on the underlying patterns in the data—can be nonlinear. + GAM can capture common nonlinear patterns that a classic linear model would miss. + GAM framework allows us to control smoothness of the predictor functions to prevent overfitting. By controlling the wiggliness of the predictor functions, we can directly tackle the bias/variance tradeoff.


Family: gaussian 
Link function: identity 

Formula:
Market.Cap ~ Total.Equity + Earnings.Before.Interest.and.Tax

Parametric coefficients:
                                  Estimate Std. Error t value Pr(>|t|)    
(Intercept)                      2.173e-09  5.850e-11   37.14   <2e-16 ***
Total.Equity                     1.052e+00  5.873e-02   17.92   <2e-16 ***
Earnings.Before.Interest.and.Tax 8.172e+00  2.327e-01   35.11   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1


Rank: 2/3
R-sq.(adj) =  0.461   Deviance explained = 43.4%
GCV = 1.1162e+20  Scale est. = 1.1147e+20  n = 1536

GAM Regression Prediction & Accuracy

RMSE:  6937151743 Error:  0.354476 R2:  0.6515361

GAM Scatter Plot 3D Visualization

  • Visualization part of gam model

  • Diagnosing GAM model issues


Method: GCV   Optimizer: magic
Model required no smoothing parameter selectionModel rank =  2 / 3 

Extreme Gradient Boosting

[17:38:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:10] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:10] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:12] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:12] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:22] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:22] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:22] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:23] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:23] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:24] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:24] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:24] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.

Extreme Gradient Boosting Prediction & Accuracy

MSE:  3.852715e+20 MAE:  18121969088  RMSE:  19628334514

[1] 3988987596
[1] 0.2038301
[1] 0.8701342
RMSE:  3988987596 Error:  0.2038301 R2:  0.8701342

Decision Tree to Trade Bank of America Stock (Real Time Value Analysis)

Building a Strategy

  • Let’s see how we can quickly build a strategy using 4 technical indicators to see whether today’s price of BoA’s stock is going to close up or down.The 4 technical indicators are:
    • RSI -> Calculate a 3-period relative strength index (RSI) off the open price
    • EMA -> Calculate a 5-period exponential moving average (EMA)
    • MACD -> Calculate a MACD with standard parameters
    • SMI -> Stochastic Oscillator with standard parameters
  • Then we calculate the variable we are looking to predict and build our data sets.
    • Class -> Calculate the difference between the close price and open price. If PriceChange>0,“UP”,“DOWN”
[1] "BAC"
[1] "RSI3"       "EMAcross"   "MACDsignal" "Stochastic" "Class"     
  • Data slicing is a step to split data into train and test set. Training data set can be used specifically for our model building. Test dataset should not be mixed up while building model. Even during standardization, we should not standardize our test set.75 percent contributes to training & 25 percent of data contributes to test.

  • Training the Decision Tree classifier with criterion as information gain

  • We are setting 3 parameters of trainControl() method. The “method” parameter holds the details about resampling method. We can set “method” to use repeatedcv i.e, repeated cross-validation.

  • The “number” parameter holds the number of resampling iterations. The “repeats” parameter contains the complete sets of folds to compute for our repeated cross-validation. We are using setting number =10 and repeats =3. This trainControl() methods returns a list. We are going to pass this on our train() method.

  • To select the specific strategy for decision tree, we need to pass a parameter “parms” in our train() method. It should contain a list of parameters for our rpart method. For splitting criterions, we need to add a “split” parameter with values either “information” for information gain & “gini” for gini index. We are using information gain as a criterion.

Trained Decision Tree classifier results:

  • We can check the result of our train() method by a print dtree_fit variable. It is showing us the accuracy metrics for different values of cp. Here, cp is complexity parameter for our dtree.
CART 

3908 samples
   4 predictor
   2 classes: 'DOWN', 'UP' 

Pre-processing: centered (4), scaled (4) 
Resampling: Cross-Validated (10 fold, repeated 3 times) 
Summary of sample sizes: 3517, 3518, 3517, 3517, 3517, 3517, ... 
Resampling results across tuning parameters:

  cp           Accuracy   Kappa        
  0.001388889  0.5141566   0.0301167276
  0.001562500  0.5145822   0.0308771188
  0.001822917  0.5152644   0.0322216484
  0.001909722  0.5154349   0.0325509692
  0.002083333  0.5159460   0.0335094058
  0.002343750  0.5170542   0.0345975203
  0.002951389  0.5177349   0.0358283997
  0.003645833  0.5188436   0.0377622413
  0.018229167  0.5058795   0.0006184276
  0.019531250  0.5053680  -0.0004969420

Accuracy was used to select the optimal model using the largest value.
The final value used for the model was cp = 0.003645833.

Plot Decision Tree

#### Decision Tree Prediction

Now, our model is trained with cp = 0.003645833.Accuracy of the model is 0.5141566.

Confusion Matrix and Statistics

          Reference
Prediction DOWN  UP
      DOWN  317 279
      UP    345 360
                                          
               Accuracy : 0.5204          
                 95% CI : (0.4928, 0.5478)
    No Information Rate : 0.5088          
    P-Value [Acc > NIR] : 0.210682        
                                          
                  Kappa : 0.0422          
                                          
 Mcnemar's Test P-Value : 0.009266        
                                          
            Sensitivity : 0.4789          
            Specificity : 0.5634          
         Pos Pred Value : 0.5319          
         Neg Pred Value : 0.5106          
             Prevalence : 0.5088          
         Detection Rate : 0.2437          
   Detection Prevalence : 0.4581          
      Balanced Accuracy : 0.5211          
                                          
       'Positive' Class : DOWN            
                                          

KNN Classifier to predict if daily stock price will increase

Building KNN model

  • Data Model which we will use for ML is listed below. Here, the 3 columns represent the closing price,opening price,High price, volume of the stock on the given date.

  • The Increase column represents whether the price of stock rose or fell as compared to the previous day.

  • We are using caret’s KNN to sample the data (for training and testing), preprocessing, evaluating the model etc.,

  • Step 1: Data Splicing

    • KNN algorithm is applied to the training data set and the results are verified on the test data set.
    • I used 25% to test data and 75% to train the data.
    • After obtaining training and testing data sets, then we will create a separate data frame from testing data set which has values to be compared with actual final values
    • Predictor variables are Open ,increase ,Volume,High. Target variable is Close
  • Step 2:Data Pre-Processing With Caret

    • We need to pre-process our data before we can use it for modeling.
    • The caret package in R provides a number of useful data transforms.
    • Training transforms can prepared and applied automatically during model evaluation.
    • Transforms applied during training are prepared using the preProcess() and passed to the train() function via the preProcess argument.
    • Combining the scale and center transforms for preprocessing will standardize the data.
    • The scale transform calculates the standard deviation for an attribute and divides each value by that standard deviation.
    • The center transform calculates the mean for an attribute and subtracts it from each value.
    • Attributes after preprocessing will have a mean value of 0 and a standard deviation of 1.
  • Step 3:Model Training and Tuning

    • To control parameters for train, trainControl function is used. “trainControl” allows estimation of parameter coefficients through resampling methods like cross validation, repeatedcv,boosting etc.
    • The option “repeatedcv” method which we used in traincontrol method controls the number of repetitions for resampling used in repeated K-fold cross-validation.
  colnames(bny_raw)
1              Date
2              Open
3          increase
4            Volume
5             Close
6              High
  • Step 4:How to choose value for K to improve performance

    • Time to fit a knn model using caret with preprocessed values.
    • From the output of the knn model ,maximum RSquared(0.9983847) is achieved by k = 5
    • Now we can plot our KNN , and view summary of the model
k-Nearest Neighbors 

8993 samples
   5 predictor
   2 classes: 'FALSE', 'TRUE' 

Pre-processing: centered (5), scaled (5) 
Resampling: Cross-Validated (10 fold, repeated 3 times) 
Summary of sample sizes: 8093, 8094, 8093, 8094, 8094, 8095, ... 
Resampling results across tuning parameters:

  k   Accuracy   Kappa     
   5  0.5257777  0.02578255
   7  0.5287424  0.02899955
   9  0.5311870  0.03274320
  11  0.5307430  0.02950820
  13  0.5333738  0.03257598
  15  0.5386393  0.04107065
  17  0.5403822  0.04288474
  19  0.5420874  0.04482209
  21  0.5460535  0.05178726
  23  0.5522058  0.06325600
  25  0.5534678  0.06464222
  27  0.5560607  0.06852801
  29  0.5552093  0.06542168
  31  0.5540608  0.06233631
  33  0.5585827  0.07069766
  35  0.5597692  0.07214944
  37  0.5595463  0.07104593
  39  0.5581399  0.06758583
  41  0.5570294  0.06480353
  43  0.5573246  0.06410129

Accuracy was used to select the optimal model using the largest value.
The final value used for the model was k = 35.

KNN Accuracy

  • Step 5: Making predictions
    • We build knn by using training & test data sets. After building the model, then we can check the accuracy of forecasting using confusion matrix.
  • Step 6: Interpretation of the results and prediction accuracy achieved.
    • The accuracy of our model on the testing set is 55%.
    • We can visualise the model’s performance using a confusion matrix.
    • We can also evaluvate the accuracy, precision and recall on the training and validation sets in confusion matrix to evaluate the performance of knn algorithm.
Confusion Matrix and Statistics

          Reference
Prediction FALSE TRUE
     FALSE   426  446
     TRUE    895 1229
                                          
               Accuracy : 0.5524          
                 95% CI : (0.5344, 0.5703)
    No Information Rate : 0.5591          
    P-Value [Acc > NIR] : 0.7748          
                                          
                  Kappa : 0.0583          
                                          
 Mcnemar's Test P-Value : <2e-16          
                                          
            Sensitivity : 0.3225          
            Specificity : 0.7337          
         Pos Pred Value : 0.4885          
         Neg Pred Value : 0.5786          
             Prevalence : 0.4409          
         Detection Rate : 0.1422          
   Detection Prevalence : 0.2911          
      Balanced Accuracy : 0.5281          
                                          
       'Positive' Class : FALSE           
                                          
Accuracy:  0.5524032

Overall insights obtained from the implemented project

  • As we can see, the model has the highest accuracy of 55 when k = 5. While this may not seem any good, it is often extremely hard to predict the price of stocks. Even the 2.5% improvement over random guessing can make a difference given the amount of money at stake. After all, if it was that easy to predict the prices, wouldn’t we all be trading in stocks for the easy money instead of learning these algorithms?
  • Sensitivity for price increase is 0.3225 .
  • Specificity for price increase is 0.7337.

K-means Clustering NYSE trades.

Modelling

             Length Class  Mode   
cluster      33333  -none- numeric
centers         14  -none- numeric
totss            1  -none- numeric
withinss         7  -none- numeric
tot.withinss     1  -none- numeric
betweenss        1  -none- numeric
size             7  -none- numeric
iter             1  -none- numeric
ifault           1  -none- numeric
Centers:  26.77171 17.36502 298.0009 84.80548 14.4589 46.8826 14.77272 8.621914 -79.9289 275.3313 64.27364 -31.47357 27.74873 -8.16676
No Of Records in Cluster:  10464 681 23 1032 5088 5087 10958

KNN Cluster Plotting

  • Plot to see how opening prices have been distributed in clusters.

Conclusion

  • KNN is a lazy learner where generalization of the training data is delayed until a query is made to the system. Which means knn starts working only when you trigger it to, thus lazy learning methods can construct a different approximation or result to the target function for each encountered query.

  • It is a good method for online learning but it requires a possibly large amount of memory to store the data, and each request involves starting the identification of a local model from scratch.

ARIMA Forecasting Expedia Stock Price incorporating COVID-19 (Real Time)

[1] "EXPE"
[1] "EXPE"
  colnames(google_data_after_covid)
1                         EXPE.Open
2                         EXPE.High
3                          EXPE.Low
4                        EXPE.Close
5                       EXPE.Volume
6                     EXPE.Adjusted

Graphical Representation of Data

ARIMA Model

  • Let us first analyse the ACF and PACF Graph of each of the two datasets.

  • We then use the auto.arima function to determine the time series model for each of the datasets.
Series: tsData_before_covid_close 
ARIMA(3,1,2) 
Box Cox transformation: lambda= -0.5522943 

Coefficients:
         ar1      ar2      ar3      ma1     ma2
      0.9436  -0.2552  -0.2793  -0.9659  0.7154
s.e.  0.1333   0.1520   0.0991   0.1174  0.1094

sigma^2 estimated as 1.264e-05:  log likelihood=791.34
AIC=-1570.69   AICc=-1570.22   BIC=-1551.3

Training set error measures:
                     ME     RMSE      MAE        MPE     MAPE     MASE
Training set -0.3468986 3.779995 2.064975 -0.4697368 2.119611 1.117624
                    ACF1
Training set -0.01611792
Series: tsData_after_covid_close 
ARIMA(1,1,0) with drift 
Box Cox transformation: lambda= 1.049021 

Coefficients:
          ar1   drift
      -0.0993  0.5238
s.e.   0.0785  0.3099

sigma^2 estimated as 19.12:  log likelihood=-470.79
AIC=947.57   AICc=947.73   BIC=956.86

Training set error measures:
                       ME     RMSE      MAE         MPE     MAPE      MASE
Training set -0.002349432 3.471361 2.361182 -0.07665327 2.827689 0.9868023
                    ACF1
Training set 0.002088858
  • From the auto.arima function, we conclude the following models for the two datasets:

    • Before COVID-19: ARIMA(3,1,2)
    • After COVID-19: ARIMA(1,1,0)
  • After obtaining the model, we then perform residual diagnostics for each of the fitted models.

  • From the residual plot , we can confirm that the residual has a mean of 0 and the variance is constant as well . The ACF is 0 for lag> 0 , and the PACF is 0 as well.

  • So, we can say that the residual behaves like white noise and conclude that the models ARIMA(3,1,2) and ARIMA(1,1,0) fits the data well. Alternatively, we can also test at a significance level using the Box-Ljung Test.

Diagnostic measures

  • Try to find out the pattern in the residuals of the chosen model by plotting the ACF of the residuals, and doing a portmanteau test. We need to try modified models if the plot doesn’t look like white noise.

  • Once the residuals look like white noise, calculate forecasts.

  • Box-Ljung test is a test of independence at all lags up to the one specified. Instead of testing randomness at each distinct lag, it tests the “overall” randomness based on a number of lags, and is therefore a portmanteau test. It is applied to the residuals of a fitted ARIMA model, not the original series, and in such applications the hypothesis actually being tested is that the residuals from the ARIMA model have no autocorrelation.

  • The ACF of the residuals shows no significant autocorrelations.

  • The p-values for the Ljung-Box Q test all are well above 0.05, indicating “non-significance.” The values are normal as they rest on a line and aren’t all over the place.

Augmented Dickey-Fuller & Kwiatkowski-Phillips-Schmidt-Shin

  • We then conduct an ADF (Augmented Dickey-Fuller) test and KPSS (Kwiatkowski-Phillips-Schmidt-Shin) test to check for the stationarity of the time series data for both the datasets closing price.

    Augmented Dickey-Fuller Test

data:  tsData_before_covid_close
Dickey-Fuller = -2.1153, Lag order = 5, p-value = 0.5279
alternative hypothesis: stationary

    Augmented Dickey-Fuller Test

data:  tsData_after_covid_close
Dickey-Fuller = -2.4665, Lag order = 5, p-value = 0.3817
alternative hypothesis: stationary
  • From the above ADF tests, we can conclude the following:

    • For the dataset before COVID-19, the ADF tests gives a p-value of 0.5279which is greater than 0.05, thus implying that the time series data is not stationary.

    • For the dataset after COVID-19, the ADF tests gives a p-value of 0.3817 which is lesser greater 0.05, thus implying that the time series data is not stationary.


    KPSS Test for Level Stationarity

data:  tsData_before_covid_close
KPSS Level = 2.3907, Truncation lag parameter = 4, p-value = 0.01

    KPSS Test for Level Stationarity

data:  tsData_after_covid_close
KPSS Level = 2.6532, Truncation lag parameter = 4, p-value = 0.01
  • From the above KPSS tests, we can conclude the following:

  • For the dataset before COVID-19, the KPSS tests gives a p-value of 0.01 which is less than 0.05, thus implying that the time series data is not stationary.

  • For the dataset after COVID-19, the KPSS tests gives a p-value of 0.01 which is less than 0.05, thus implying that the time series data is not stationary.

  • Thus, we can conclude from the above tests that the time series data is not stationary.

  • Forecasting with ARIMA Models

  • Forecast errors for before covid dataset

  • Forecast errors for after covid dataset

  • Holt Winters

    • To make forecasts using simple exponential smoothing in R, we can fit a simple exponential smoothing predictive model using the “HoltWinters()” function in R. To use HoltWinters() for simple exponential smoothing, we need to set the parameters beta=FALSE and gamma=FALSE in the HoltWinters() function (the beta and gamma parameters are used for Holt’s exponential smoothing, or Holt-Winters exponential smoothing, as described below).

    • HoltWinters() function returns a list variable, that contains several named elements.The output of HoltWinters() tells us that the estimated value of the alpha parameter is about 0.982813.

    • We can plot the original time series.

Holt-Winters exponential smoothing with trend and without seasonal component.

Call:
HoltWinters(x = tsData_before_covid_close, gamma = FALSE)

Smoothing parameters:
 alpha: 0.982813
 beta : 0.02381423
 gamma: FALSE

Coefficients:
       [,1]
a 60.091954
b -0.869578

  • As a measure of the accuracy of the forecasts, we can calculate the sum of squared errors for the in-sample forecast errors, that is, the forecast errors for the time period covered by our original time series. The sum-of-squared-errors is stored in a named element of the list variable “rainseriesforecasts” called “SSE”, so we can get its value by typing:
[1] "skirtsseriesforecasts$SSE"
[1] 2798.291

Forecasting with HoltWinters

  • We can make forecasts for further time points by using the “forecast.HoltWinters()” function in the R “forecast” package.
  • When using the forecast.HoltWinters() function, as its first argument (input), you pass it the predictive model that you have already fitted using the HoltWinters() function.
  • You specify how many further time points you want to make forecasts for by using the “h” parameter in forecast.Here we are going to predict price for 45 days.
  • forecast.HoltWinters() function gives you the forecast for a year, a 80% prediction interval for the forecast, and a 95% prediction interval for the forecast. For example, the forecasted stock price for 233 day is about 20.96095, with a 95% prediction interval of (-56.8291332 98.75102).
    Point Forecast      Lo 80    Hi 80       Lo 95    Hi 95
189       59.22238  54.261714 64.18304  51.6356981 66.80905
190       58.35280  51.315518 65.39008  47.5902068 69.11539
191       57.48322  48.787761 66.17868  44.1846635 70.78178
192       56.61364  46.469767 66.75752  41.0999238 72.12736
193       55.74406  44.280562 67.20757  38.2121512 73.27598
194       54.87449  42.178858 67.57011  35.4581988 74.29077
195       54.00491  40.140333 67.86948  32.8008698 75.20895
196       53.13533  38.149294 68.12137  30.2161653 76.05449
197       52.26575  36.194958 68.33655  27.6875932 76.84391
198       51.39617  34.269562 68.52279  25.2032811 77.58907
199       50.52660  32.367315 68.68588  22.7543724 78.29882
200       49.65702  30.483773 68.83026  20.3340704 78.97997
201       48.78744  28.615447 68.95943  17.9370389 79.63784
202       47.91786  26.759545 69.07618  15.5590078 80.27672
203       47.04828  24.913796 69.18277  13.1965057 80.90006
204       46.17871  23.076330 69.28108  10.8466714 81.51074
205       45.30913  21.245588 69.37267   8.5071196 82.11114
206       44.43955  19.420256 69.45884   6.1758412 82.70326
207       43.56997  17.599217 69.54073   3.8511288 83.28882
208       42.70039  15.781515 69.61927   1.5315205 83.86927
209       41.83082  13.966326 69.69531  -0.7842448 84.44588
210       40.96124  12.152934 69.76954  -3.0972610 85.01974
211       40.09166  10.340716 69.84261  -5.4084829 85.59180
212       39.22208   8.529124 69.91504  -7.7187474 86.16291
213       38.35250   6.717675 69.98733 -10.0287917 86.73380
214       37.48293   4.905945 70.05991 -12.3392679 87.30512
215       36.61335   3.093553 70.13314 -14.6507544 87.87745
216       35.74377   1.280164 70.20738 -16.9637667 88.45131
217       34.87419  -0.534523 70.28291 -19.2787648 89.02715
218       34.00461  -2.350779 70.36001 -21.5961611 89.60539
219       33.13504  -4.168844 70.43892 -23.9163256 90.18640
220       32.26546  -5.988937 70.51985 -26.2395913 90.77051
221       31.39588  -7.811255 70.60302 -28.5662585 91.35802
222       30.52630  -9.635974 70.68858 -30.8965985 91.94920
223       29.65672 -11.463254 70.77670 -33.2308566 92.54431
224       28.78715 -13.293243 70.86754 -35.5692552 93.14355
225       27.91757 -15.126070 70.96121 -37.9119963 93.74713
226       27.04799 -16.961857 71.05784 -40.2592630 94.35524
227       26.17841 -18.800712 71.15754 -42.6112221 94.96805
228       25.30884 -20.642734 71.26040 -44.9680254 95.58570
229       24.43926 -22.488014 71.36653 -47.3298109 96.20832
230       23.56968 -24.336634 71.47599 -49.6967045 96.83606
231       22.70010 -26.188669 71.58887 -52.0688212 97.46902
232       21.83052 -28.044188 71.70523 -54.4462656 98.10731
233       20.96095 -29.903253 71.82514 -56.8291332 98.75102
  • To plot the predictions made by forecast.HoltWinters(), we can use the “plot.forecast()” function:
  • Prediction for next ten days.

  • Ljung-Box Tests

  • Box Test for modelfit_after_covid$residuals.

  • To check for correlations between successive forecast errors, we can make a correlogram and use the Ljung-Box test.


    Box-Ljung test

data:  modelfit_after_covid$residuals
X-squared = 1.6058e-05, df = 1, p-value = 0.9968
  • Box Test for modelfit_before_covid$residuals

    Box-Ljung test

data:  modelfit_before_covid$residuals
X-squared = 0.0047812, df = 1, p-value = 0.9449

Conclusion

Here, the p value for both the models is greater than 0.05 . Hence, at a significance level of 0.05 we fail to reject the null hypothesis and conclude that the residual follows white noise. This means that the model fits the data well.

Using a Neural Network to Model the Amazon Stock Price Change (Real Time Analysis)

[1] "AMZN"

Normalize Data

  • One of the most important procedures when forming a neural network is data normalization. This involves adjusting the data to a common scale so as to accurately compare predicted and actual values. Failure to normalize the data will typically result in the prediction value remaining the same across all observations, regardless of the input values.

  • Max-Min Normalization: For this method, we invoke the following function to normalize our data:

  • Data slicing:

    • Data slicing is a step to split data into train and test set. Training data set can be used specifically for our model building. Test dataset should not be mixed up while building model. Even during standardization, we should not standardize our test set.75 percent contributes to training & 25 percent of data contributes to test.
'data.frame':   5209 obs. of  5 variables:
 $ RSI3      : num  0.0853 0.0409 0.0402 0.6396 0.4188 ...
 $ EMAcross  : num  0.538 0.534 0.539 0.553 0.545 ...
 $ MACDsignal: num  0.727 0.705 0.681 0.661 0.643 ...
 $ BollingerB: num  0.423 0.353 0.362 0.477 0.408 ...
 $ Price     : num  0.417 0.424 0.444 0.422 0.433 ...
  • TrainingParameters :
    • train() method is passed with repeated cross-validation resampling method for 10 number of resampling iterations repeated for 3 times.

Machine Learning: Classification using Neural Networks

  • Model Training
    • We can us neuralnet() to train a NN model. Also, the train() function from caret can help us tune parameters. We can plot the result to see which set of parameters is fit our data the best.
    • nnnet package by defualt uses the Logisitc Activation function.
    • Data Pre-Processing With Caret: The scale transform calculates the standard deviation for an attribute and divides each value by that standard deviation.
    • The center transform calculates the mean for an attribute and subtracts it from each value.
    • Combining the scale and center transforms will standardize your data.
    • Attributes will have a mean value of 0 and a standard deviation of 1.
    • Training transforms can prepared and applied automatically during model evaluation.
    • Transforms applied during training are prepared using the preProcess() and passed to the train() function via the preProcess argument.
    • Backpropagation algorithm is a supervised learning method for multilayer feed-forward networks from the field of Artificial Neural Networks.
    • The principle of the backpropagation approach is to model a given function by modifying internal weightings of input signals to produce an expected output signal. The system is trained using a supervised learning method, where the error between the system’s output and a known expected output is presented to the system and used to modify its internal state.
    • We use Backpropagation as algorithm in neural network package.
nnetGrid <-  expand.grid(size = seq(from = 1, to = 5, by = 1)
                         ,decay = seq(from = 0.1, to = 0.2, by = 0.1)
)
str(subTrain)
'data.frame':   12507 obs. of  5 variables:
 $ RSI3      : num  0.0853 0.0409 0.0402 0.4188 0.3651 ...
 $ EMAcross  : num  0.538 0.534 0.539 0.545 0.544 ...
 $ MACDsignal: num  0.727 0.705 0.681 0.643 0.625 ...
 $ BollingerB: num  0.423 0.353 0.362 0.408 0.386 ...
 $ Price     : num  0.417 0.424 0.444 0.433 0.426 ...
nn_model <- train(Price~RSI3+EMAcross+MACDsignal+BollingerB, subTrain,
                  method = "nnet",  algorithm = 'backprop',     
                  trControl= TrainingParameters,
                  preProcess=c("scale","center"),
                  na.action = na.omit,
                  tuneGrid = nnetGrid,
                  trace=FALSE,
                  verbose=FALSE)      
  • Based on the caret neural network model, train sets hidden layer.caret neural network picks the best neural network based on size, decay.
    • We can visualize accuracy for different hidden layers below:
   size decay       RMSE   Rsquared        MAE      RMSESD RsquaredSD
1     1   0.1 0.03933584 0.01238798 0.01586244 0.002504076 0.01443134
2     1   0.2 0.03934006 0.01239107 0.01583777 0.002507265 0.01451383
3     2   0.1 0.03933346 0.01268185 0.01586449 0.002503906 0.01468389
4     2   0.2 0.03933083 0.01281302 0.01584319 0.002506894 0.01488929
5     3   0.1 0.03933819 0.01287279 0.01586899 0.002504096 0.01467428
6     3   0.2 0.03933054 0.01293707 0.01584766 0.002504418 0.01501688
7     4   0.1 0.03932220 0.01314280 0.01584623 0.002496180 0.01459575
8     4   0.2 0.03932984 0.01300564 0.01584778 0.002504590 0.01508909
9     5   0.1 0.03932381 0.01361986 0.01584481 0.002522508 0.01473453
10    5   0.2 0.03932938 0.01304931 0.01584684 0.002504771 0.01512491
          MAESD
1  0.0007305646
2  0.0007313067
3  0.0007312625
4  0.0007335536
5  0.0007326783
6  0.0007321606
7  0.0007377006
8  0.0007321347
9  0.0007322239
10 0.0007327730

  • Prediction
    • Now, our model is trained with accuracy = 0.8889 We are ready to predict classes for our test set.
        actual prediction
7    0.4362379  0.4274795
10   0.4256909  0.4273008
97   0.4278731  0.4284938
139  0.4238725  0.4280213
483  0.4315391  0.4283605
783  0.4282804  0.4271653
896  0.4283095  0.4268096
908  0.4300552  0.4267116
911  0.4315391  0.4280867
952  0.4203374  0.4286907
1012 0.4257782  0.4279932
1711 0.4255164  0.4265486
1746 0.4299679  0.4280298
1804 0.4386092  0.4280147
2058 0.4317427  0.4280102
2073 0.4374163  0.4289039
2176 0.4056444  0.4292499
2276 0.4372417  0.4271038
2695 0.4251672  0.4280948
2746 0.4161477  0.4282272
2772 0.4228978  0.4272442
2784 0.4386092  0.4272447
3057 0.4236834  0.4273921
3139 0.4282513  0.4273274
3178 0.4408495  0.4281902
3220 0.4289496  0.4283887
3335 0.4316845  0.4266736
3443 0.4418097  0.4276801
3503 0.4245853  0.4275239
4018 0.4396857  0.4311302
4237 0.4353504  0.4281576
4261 0.4384055  0.4282462
4421 0.4145185  0.4284854
4526 0.5582194  0.4272961
4664 0.4444863  0.4258982
4786 0.4411985  0.4281155
4843 0.3776257  0.4279618
4865 0.4139076  0.4277111
4937 0.4610998  0.4286968
5030 0.5170493  0.4270784

Accuracy of Neural Network model:

99% accuracy is acheived to predict the price using the stock technical indicators.

           RSI3  EMAcross MACDsignal BollingerB     Price
7    0.32459911 0.5442847  0.6075494  0.3572573 0.4362379
10   0.20618845 0.5412815  0.5634201  0.3204323 0.4256909
97   0.24607269 0.5460137  0.2333221  0.3782227 0.4278731
139  0.84671114 0.5597144  0.7114260  0.7928638 0.4238725
483  0.80588690 0.5503068  0.6449523  0.7104961 0.4315391
783  0.54896975 0.5481855  0.7942074  0.5794676 0.4282804
896  0.36179313 0.5467044  0.7485756  0.4977961 0.4283095
908  0.36059256 0.5461656  0.7215884  0.5917109 0.4300552
911  0.87072321 0.5542228  0.7250983  0.8051809 0.4315391
952  0.72799440 0.5495230  0.5471357  0.5809809 0.4203374
1012 0.25508071 0.5462964  0.4156671  0.3042368 0.4257782
1711 0.17516899 0.5466259  0.7149182  0.3455139 0.4255164
1746 0.42878584 0.5473307  0.5385448  0.3833825 0.4299679
1804 0.95547536 0.5649359  0.7302750  0.9376553 0.4386092
2058 0.75475738 0.5528730  0.6755122  0.7233058 0.4317427
2073 0.85834756 0.5557843  0.6024193  0.5896256 0.4374163
2176 0.61647522 0.5560750  0.3465414  0.4212848 0.4056444
2276 0.38169960 0.5460403  0.7248211  0.4346285 0.4372417
2695 0.79630557 0.5568765  0.7103222  0.6807527 0.4251672
2746 0.86740868 0.5570476  0.6578077  0.8334958 0.4161477
2772 0.33325260 0.5425081  0.6137948  0.4871775 0.4228978
2784 0.11853439 0.5357345  0.5229431  0.2911652 0.4386092
3057 0.37948310 0.5450882  0.6474615  0.4269868 0.4236834
3139 0.47640198 0.5473919  0.6464690  0.6264217 0.4282513
3178 0.64602849 0.5532853  0.6476852  0.4856036 0.4408495
3220 0.78578515 0.5543646  0.6158716  0.6913156 0.4289496
3335 0.14403635 0.5345840  0.5990995  0.4992964 0.4316845
3443 0.95580068 0.6123559  0.6557729  0.9230118 0.4418097
3503 0.48426281 0.5469840  0.6377598  0.5614055 0.4245853
4018 0.04419073 0.4504553  0.4264717  0.1988550 0.4396857
4237 0.59029580 0.5558939  0.5694519  0.5221716 0.4353504
4261 0.87353837 0.5836426  0.6483177  0.7248558 0.4384055
4421 0.88473691 0.5823825  0.5743706  0.7673681 0.4145185
4526 0.59590136 0.5793888  0.7331421  0.5560403 0.5582194
4664 0.98636455 0.6589197  0.6674518  0.8129811 0.4444863
4786 0.62740722 0.5626872  0.5948031  0.5374183 0.4411985
4843 0.54126423 0.5618588  0.6234117  0.4116830 0.3776257
4865 0.63945676 0.5788562  0.5987941  0.6760626 0.4139076
4937 0.20789866 0.4876023  0.5509471  0.2674240 0.4610998
5030 0.44684424 0.5332037  0.7226938  0.6420079 0.5170493
 [1]  0.0034655906 -0.0006396635 -0.0002464325 -0.0016497116  0.0012600367
 [6]  0.0004426085  0.0005953472  0.0013262634  0.0013685815 -0.0033262027
[11] -0.0008800795 -0.0004101826  0.0007687922  0.0041881569  0.0014795282
[16]  0.0033666456 -0.0094548054  0.0040098370 -0.0011635185 -0.0048179652
[21] -0.0017289311  0.0044925247 -0.0014748155  0.0003667483  0.0049999668
[26]  0.0002225924  0.0019863283  0.0055785501 -0.0011681378  0.0033806843
[31]  0.0028470762  0.0040164223 -0.0055743799  0.0494191079  0.0073310991
[36]  0.0051666092 -0.0203900965 -0.0055105203  0.0126964783  0.0344970986
[1] 0.9974791
  • Plotting nnet variable importance
nnet variable importance

           Overall
EMAcross   100.000
MACDsignal  11.556
BollingerB   6.033
RSI3         0.000

#### Graphical Representation of our Neural Network

Conclusion

From the above implementation, the results are impressive(99% accuracy) and convincing in terms of using a machine learning algorithm to decide on the price of the stock Majority of the attributes in the dataset contribute significantly to the building of a predictive model.

Using SVM to predict price change for WALMART (Real Time Analysis)

Machine Learning: Classification using SVM

  • SVM is another classification method that can be used to predict if a client falls into either ‘yes’ or ‘no’ class.
    • The linear, polynomial and RBF or Gaussian kernel in SVM are simply different in case of making the hyperplane decision boundary between the classes.
    • The kernel functions are used to map the original dataset (linear/nonlinear ) into a higher dimensional space with view to making it linear dataset.
    • Usually linear and polynomial kernels are less time consuming and provides less accuracy than the rbf or Gaussian kernels.
    • The k cross validation is used to divide the training set into k distinct subsets. Then every subset is used for training and others k-1 are used for validation in the entire trainging phase. This is done for the better training of the classification task.Overall, if you are unsure which kernel method would be best, a good practice is use of something like 10-fold cross-validation for each training set and then pick the best algorithm.
   colnames(DataSetWalmart)
1                     Class
2                       RSI
3                  EMAcross
4                      MACD
5                       SMI
6                       WPR
7                       ADX
8                       CCI
9                       CMO
10                      ROC

SVM Classifier using Linear Kernel

  • Caret package provides train() method for training our data for various algorithms. We just need to pass different parameter values for different algorithms. Before train() method, we will first use trainControl() method.

    • We are setting 3 parameters of trainControl() method. The “method” parameter holds the details about resampling method. We can set “method” with many values like “boot”, “boot632”, “cv”, “repeatedcv”, “LOOCV”, “LGOCV” etc. For this project, let’s try to use repeatedcv i.e, repeated cross-validation.

    • The “number” parameter holds the number of resampling iterations. The “repeats” parameter contains the complete sets of folds to compute for our repeated cross-validation. We are using setting number =10 and repeats =3. This trainControl() methods returns a list. We are going to pass this on our train() method.

    • Before training our SVM classifier, set.seed().

    • For training SVM classifier, train() method should be passed with “method” parameter as “svmLinear”. We are passing our target variable Term_Deposit. The “Term_Deposit.~.” denotes a formula for using all attributes in our classifier and Term_Deposit. as the target variable. The “trControl” parameter should be passed with results from our trianControl() method. The “preProcess” parameter is for preprocessing our training data.

    • As discussed earlier for our data, preprocessing is a mandatory task. We are passing 2 values in our “preProcess” parameter “center” & “scale”. These two help for centering and scaling the data. After preProcessing these convert our training data with mean value as approximately “0” and standard deviation as “1”. The “tuneLength” parameter holds an integer value. This is for tuning our algorithm.

Support Vector Machines with Linear Kernel 

18651 samples
    7 predictor
    2 classes: 'DOWN', 'UP' 

Pre-processing: centered (7), scaled (7) 
Resampling: Cross-Validated (10 fold, repeated 1 times) 
Summary of sample sizes: 16786, 16786, 16786, 16786, 16786, 16786, ... 
Resampling results across tuning parameters:

  C     Accuracy   Kappa    
  0.25  0.8209209  0.6412469
  0.50  0.8211354  0.6416789

Accuracy was used to select the optimal model using the largest value.
The final value used for the model was C = 0.5.
  • The above model is showing that our classifier is giving best accuracy on C = 0.25 Let’s try to make predictions using this model for our test set and check its accuracy.

             
predictionsvm DOWN UP
         DOWN   28  8
         UP      5 28
  • Accuracy on the test set by train control is 81% using C=0.25.
[1] 0.8115942
Confusion Matrix and Statistics

          Reference
Prediction DOWN UP
      DOWN   28  8
      UP      5 28
                                          
               Accuracy : 0.8116          
                 95% CI : (0.6994, 0.8957)
    No Information Rate : 0.5217          
    P-Value [Acc > NIR] : 5.253e-07       
                                          
                  Kappa : 0.6239          
                                          
 Mcnemar's Test P-Value : 0.5791          
                                          
            Sensitivity : 0.8485          
            Specificity : 0.7778          
         Pos Pred Value : 0.7778          
         Neg Pred Value : 0.8485          
             Prevalence : 0.4783          
         Detection Rate : 0.4058          
   Detection Prevalence : 0.5217          
      Balanced Accuracy : 0.8131          
                                          
       'Positive' Class : DOWN            
                                          

SVM Classifier using Non-Linear Kernel

  • Now, we will try to build a model using Non-Linear Kernel like Radial Basis Function. For using RBF kernel, we just need to change our train() method’s “method” parameter to “svmRadial”. In Radial kernel, it needs to select proper value of Cost “C” parameter and “sigma” parameter.
Support Vector Machines with Radial Basis Function Kernel 

18651 samples
    7 predictor
    2 classes: 'DOWN', 'UP' 

Pre-processing: centered (7), scaled (7) 
Resampling: Cross-Validated (10 fold, repeated 1 times) 
Summary of sample sizes: 16786, 16786, 16786, 16786, 16786, 16786, ... 
Resampling results across tuning parameters:

  C     Accuracy   Kappa    
  0.25  0.8565757  0.7127826
  0.50  0.8618302  0.7233515

Tuning parameter 'sigma' was held constant at a value of 0.5
Accuracy was used to select the optimal model using the largest value.
The final values used for the model were sigma = 0.5 and C = 0.5.
  • SVM-RBF kernel calculates variations and will present us best values of sigma & C. Based on the output best values of sigma= 0.5 & C=0.5 Let’s check our trained models’ accuracy on the test set.
[1] 0.7971014
  • Final prediction accuracy on the test set is 0.7971014

Comparision between SVM models

  • Comparision between SVM Linear and Radial Models.

Call:
summary.resamples(object = algo_results)

Models: SVM_RADIAL, SVM_LINEAR 
Number of resamples: 10 

Accuracy 
                Min.   1st Qu.    Median      Mean   3rd Qu.      Max. NA's
SVM_RADIAL 0.8509383 0.8571046 0.8630027 0.8618302 0.8657375 0.8723861    0
SVM_LINEAR 0.8144772 0.8171582 0.8209115 0.8211354 0.8248692 0.8284182    0

Kappa 
                Min.   1st Qu.    Median      Mean   3rd Qu.      Max. NA's
SVM_RADIAL 0.7011356 0.7139651 0.7255579 0.7233515 0.7312067 0.7446252    0
SVM_LINEAR 0.6280912 0.6334363 0.6410743 0.6416789 0.6492305 0.6567207    0

Conclusion

From the above implementation, the results are impressive and convincing in terms of using a machine learning algorithm to decide on the price change of walmart . Majority of the attributes in the dataset contribute significantly to the building of a predictive model. All the two SVM approach acheives good accuracy rate(>80%) and are easier to implement.

Naive Bayes Sentimental Analysis of Twitter Data for Apple Stock

To collect the data needed for this task, we had to perform two steps.

The following graph shows the distribution of the number of tweets classified into each of the categories. We can see here that the majority of tweets were classified as neutral, with a small number classified as strongly negative or strongly positive.

                                                                                                  Tweet
1 I have to say, Apple has by far the best customer care service I have ever received! @Apple @AppStore
2                                          iOS 7 is so fricking smooth & beautiful!! #ThanxApple @Apple
3                                                                                         LOVE U @APPLE
4           Thank you @apple, loving my new iPhone 5S!!!!!  #apple #iphone5S pic.twitter.com/XmHJCU4pcb
5                    .@apple has the best customer service. In and out with a new phone in under 10min!
6                         @apple ear pods are AMAZING! Best sound from in-ear headphones I've ever had!
  Avg
1 2.0
2 2.0
3 1.8
4 1.8
5 1.8
6 1.8

Creating Corpus

  • One of fundamental concepts in text analysis, implemented in the package tm as well, is that of a corpus.A corpus is a collection of documents.

  • We will need to convert our tweets to a corpus for pre-processing. Various function in the tm package can be used to create a corpus in many different ways.

  • We will create it from the tweet column of our data frame using two functions, Corpus() and VectorSource(). We feed to this latter the Tweets variable of the tweets data frame.

  • Pre-processing steps

    • To deal with text data following pre-processing is required.Follow the standard steps to build and pre-process the corpus:

      • Build a new corpus variable called corpus.
      • Using tm_map, convert the text to lowercase.
      • Using tm_map, remove all punctuation from the corpus.
      • Using tm_map, remove all English stopwords from the corpus.
      • Using tm_map, stem the words in the corpus.
    • Build a document term matrix from the corpus, called dtm. Each operation, like stemming or removing stop words, can be done with one line in R, where we use the tm_map() function which takes as its first argument the name of a corpus and as second argument a function performing the transformation that we want to apply to the text.

Positive Bag Of Words

  • Create a Document Term Matrix
  • We are now ready to extract the word frequencies to be used in our prediction problem. The tm package provides a function called DocumentTermMatrix() that generates a matrix where:
    • the rows correspond to documents, in our case tweets, and
    • the columns correspond to words in those tweets.
    • The values in the matrix are the number of times that word appears in each document.
  • Create word cloud for postive tweets
'data.frame':   61 obs. of  1 variable:
 $ text: chr  " say appl far best custom care servic ever receiv appl appstor" "io frick smooth beauti thanxappl appl" "love u appl" "thank appl love new iphon s appl iphon pictwittercomxmhjcupcb" ...

Negative Bag Of Words & wordcloud for negative

<<DocumentTermMatrix (documents: 1181, terms: 3198)>>
Non-/sparse entries: 10102/3766736
Sparsity           : 100%
Maximal term length: 99
Weighting          : term frequency (tf)
Sample             :
      Terms
Docs   appl get ipad iphon ipod itun make new phone whi
  103     1   0    0     0    0    0    0   0     0   0
  1066    2   0    1     0    0    1    0   0     0   0
  215     3   0    2     0    0    0    0   0     0   0
  506     3   0    0     0    0    0    0   0     0   0
  592     1   0    0     0    0    0    0   0     1   1
  62      1   1    0     0    0    0    0   0     1   1
  740     1   1    0     0    0    0    0   1     0   0
  756     1   0    0     0    0    0    0   0     0   0
  86      1   0    0     2    0    0    0   1     0   0
  900     1   0    0     0    0    0    0   0     0   0

  • Building machine learning model

    • Before Building the machine learning model, we need to split our data in training and training dataset split data in training/testing sets
    • Lastly, let’s split our data into a training set and a testing set, putting 70% of the data in the training set.
  • Making predictions

    • We build Naive Bayes by using training & test data sets.
    • We apply Laplace smoothing , which is a technique for smoothing categorical data.
    • A small-sample correction, or pseudo-count, will be incorporated in every probability estimate. Consequently, no probability will be zero. this is a way of regularizing Naive Bayes, and when the pseudo-count is zero, it is called Laplace smoothing.
model <- naive_bayes(as.data.frame(as.matrix(dtm_train)), dataset_train$positive, laplace = 1)

Interpretation of the results and prediction accuracy achieved

  • Evaluate the model performance using confusionMatrix
  • The accuracy of our model on the testing set is 78%.
  • We can visualise the model’s performance using a confusion matrix.
  • We can evaluvate the accuracy, precision and recall on the training and validation sets to evaluate the performance of naive bayes algorithm.
Confusion Matrix and Statistics

          Reference
Prediction FALSE TRUE
     FALSE   237    0
     TRUE     65    0
                                          
               Accuracy : 0.7848          
                 95% CI : (0.7341, 0.8298)
    No Information Rate : 1               
    P-Value [Acc > NIR] : 1               
                                          
                  Kappa : 0               
                                          
 Mcnemar's Test P-Value : 2.051e-15       
                                          
            Sensitivity : 0.7848          
            Specificity :     NA          
         Pos Pred Value :     NA          
         Neg Pred Value :     NA          
             Prevalence : 1.0000          
         Detection Rate : 0.7848          
   Detection Prevalence : 0.7848          
      Balanced Accuracy :     NA          
                                          
       'Positive' Class : FALSE           
                                          
  • Evaluate the model performance using CrossTable

 
   Cell Contents
|-------------------------|
|                       N |
|-------------------------|

 
Total Observations in Table:  302 

 
              | dataset_test$positive 
model_predict |     FALSE | Row Total | 
--------------|-----------|-----------|
        FALSE |       237 |       237 | 
--------------|-----------|-----------|
         TRUE |        65 |        65 | 
--------------|-----------|-----------|
 Column Total |       302 |       302 | 
--------------|-----------|-----------|

 
  • Find 25 frequent words
[1] "appl"  "iphon" "new"   "phone" "whi"   "get"   "ipad"  "itun"  "ipod" 

Overall insights obtained from the implemented project

  • Overall accuracy of the model is 78%.It is safe to assume that naive bayes models can be trained on to find the sentiment of the stocks.
  • Sensitivity for finding ratings is 0.7848.
  • Since the dataset was clean, and reviews are equally distributed between test & training set, adding laplace smoothing factor did not make much difference in the accuracy.

Amazon Price Trend Predict - Random Forest (Real Time Analysis)

Methadology

[1] "^GSPC"

  • Dataset used for this project:
  colnames(dataset1)
1                rsi
2                EMA
3             signal
4               pctB
5         GSPC.Close
  • Check for missing data in relativeStrengthIndex20, exponentialMovingAverage20, MACDsignal, PercentageChngpctB, Price. Omit the n/a values in dataset.

  • Print the number of missed value for each attribute in dataset.

[1] "rsi"
[1] 20
[1] "EMA"
[1] 19
[1] "signal"
[1] 33
[1] "pctB"
[1] 19
[1] "GSPC.Close"
[1] 0
  • RandoM Forest Machine Learning Model

    • Split the dataset into training & test dataset.80 % of the data is training data.20 % of the data is test dataset.

    • Feature Scaling -> Normalization /Scale and dropping the feature varaibles.

# Feature Scaling (Normalization and dropping the predicted variable)
training_set[-5] = scale(training_set[-5])
test_set[-5] = scale(test_set[-5])
  • Applying Random Forest Model on the Training set.
                Length Class  Mode     
call               4   -none- call     
type               1   -none- character
predicted       1884   factor numeric  
err.rate          30   -none- numeric  
confusion          6   -none- numeric  
votes           3768   matrix numeric  
oob.times       1884   -none- numeric  
classes            2   -none- character
importance         4   -none- numeric  
importanceSD       0   -none- NULL     
localImportance    0   -none- NULL     
proximity          0   -none- NULL     
ntree              1   -none- numeric  
mtry               1   -none- numeric  
forest            14   -none- list     
y               1884   factor numeric  
test               0   -none- NULL     
inbag              0   -none- NULL     

Prediction & Accuracy.

  • After building the model, then we can check the accuracy of forecasting using confusion matrix.
    • Final Model Accuracy of the model is 51 percent.
   predict_val
      0   1
  0  89 128
  1 102 152
[1] "Model Accuracy is"
[1] 0.5116773

Ensemble Method for Random Forest

Random Forest 

1884 samples
   4 predictor
   2 classes: '0', '1' 

No pre-processing
Resampling: Cross-Validated (3 fold) 
Summary of sample sizes: 1257, 1256, 1255 
Resampling results across tuning parameters:

  mtry  Accuracy   Kappa       
  3     0.5058141  -0.003131317
  6     0.5021045  -0.009128875
  9     0.5132502   0.013836823

Kappa was used to select the optimal model using the largest value.
The final value used for the model was mtry = 9.

Overall insights obtained from the implemented project

  • As we can see, the model has the highest accuracy of 51 percent . While this may not seem any good, it is often extremely hard to predict the price of stocks. Even the 2.5% improvement over random guessing can make a difference given the amount of money at stake. After all, if it was that easy to predict the prices, wouldn’t we all be trading in stocks for the easy money instead of learning these algorithms?

Discussion and Conclusions